Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Improved elastic network model for deep neural network
FENG Minghao, ZHANG Tianlun, WANG Linhui, CHEN Rong, LIAN Shaojing
Journal of Computer Applications    2019, 39 (10): 2809-2814.   DOI: 10.11772/j.issn.1001-9081.2019040624
Abstract458)      PDF (886KB)(364)       Save
Deep neural networks tend to suffer from overfitting problem because of the high complexity of the model. To reduce the adverse eeffects of the problem on the network performance, an improved elastic network model based deep learning optimization method was proposed. Firstly, considering the strong correlation between the variables, the adaptive weights were assigned to different variables of L1-norm in elastic network model, so that the linear combination of the L2-norm and the adaptively weighted L1-norm was obtained. Then, the solving process of neural network parameters under this new regularization term was given by combining improved elastic network model with the deep learning optimization model. Moreover, the robustness of this proposed model was theoretically demonstrated by showing the grouping selection ability and Oracle property of the improved elastic network model in the optimization of neural network. At last, in regression and classification experiments, the proposed model was compared with L1-norm, L2-norm and elastic network regularization term, and had the regression error decreased by 87.09, 88.54 and 47.02 and the classification accuracy improved by 3.98, 2.92 and 3.58 percentage points respectively. Thus, theory and experimental results prove that the improved elastic network model can effectively improve the generalization ability of deep neural network model and the performance of optimization algorithm, and solve the overfitting problem of deep learning.
Reference | Related Articles | Metrics